Papers with fully-connected attention
Link Prediction on N-ary Relational Facts: A Graph-based Approach (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing work on knowledge graphs (KGs) focused on binary relations, but higher-arity relations are ubiquitous in real-world KGs. |
| Approach: | They propose a graph-based approach to link prediction on knowledge graphs using n-ary relational facts and edge-biased fully-connected attention. |
| Outcome: | The proposed approach performs substantially better than current state-of-the-art across a variety of n-ary relational benchmarks. |
Star-Transformer (N19-1)
Copied to clipboard
| Challenge: | Existing models with fully-connected attention connections are heavy and require large training data. |
| Approach: | They propose a lightweight alternative to the Transformer by sparsifying the fully-connected structure with a star-shaped topology. |
| Outcome: | The proposed model achieves significant performance improvements on 22 datasets on four tasks. |